PP-ICONS-Another Tool to Help Interpret Asthma Utilization Studies

Interpret Asthma Utilization Studies When skimming through the piles of journals that arrive in your mailbox every month, do you have a quick method to bookmark high-quality studies for later review? Can you read the abstract and essentially “judge a book by its cover?” A model was recently published to do exactly that. Starting with the traditional evidence-based medicine “PICO” model of framing a clinical question, the author expanded the idea into “PP-ICONS” and applied it to evaluation of clinical literature (Table 1). Given the large number of potential studies we can read each month, it is critical to spend our time on those that are most relevant and valid; therefore, if the abstract fails to meet the PP-ICONS criteria, our time may be better spent elsewhere. Obviously, no tool is perfect in every situation, and PP-ICONS is no exception. While being better suited to treatment and prevention studies, it can also be used in diagnostic and screening studies. If we desire to test-drive our new tool and look at the asthma health care utilization study by Allen-Ramey et al. in this issue of JMCP, how does the abstract perform? The problem presented is one faced by almost any large managed care organization or solo physician practice—asthma. The patients were aged 4 to 55 years and pulled from a proprietary administrative claims database. They were required to have an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code for asthma on a medical or facility claim, or at least 2 pharmacy claims for an asthma medication. Since the study was retrospective, it was difficult to control for all relevant patient characteristics, such as disease severity. In order to account for different characteristics (covariates), a propensity model was created to match patients evenly into both treatment groups. The intervention tested was inhaled corticosteroids (ICSs) and montelukast (MON), while the active comparator was ICSs and salmeterol (SAL). Of immediate concern is the time period selected by the authors: 1998-1999. During this time, the combination product Advair (fluticasone/salmeterol) was not yet available; hence, both treatment groups required 2 separate medications. In 2006, a typical ICS/SAL prescription would be for a single product—Advair—that would very likely have superior compliance to the dual-product regimen required with ICS/MON. Secondly, the study design presumes that both ICS/MON and ICS/SAL are equivalent regimens for asthma control. This decision, unfortunately, is dubious since both the 1997 and 2002 asthma guidelines from the National Heart, Lung, and Blood Institute (NHLBI) clearly state that ICS/SAL is the preferred treatment choice when ICS alone has failed, particularly for moderate persistent and severe persistent asthma. The 2002 guideline update notes, “... there are two preferred options for treating moderate asthma: either the addition of long-acting inhaled beta2-agonists to a low dose of inhaled corticosteroids or medium-dose inhaled corticosteroids as monotherapy.” The outcomes tracked by the authors were oral corticosteroid (OCS) fill rates, short-acting beta-agonist (SABA) fill rates, visits to the emergency department (ED), and asthma-related hospitalizations. Since these outcomes are patient-oriented, and SABA overuse is a common marker used by clinicians to monitor for poor disease control, the outcomes are relevant and well chosen. When reviewing the outcome results, a significant discrepancy is immediately noted. The SABA fill rate was significantly higher among ICS/MON patients than ICS/SAL (adjusted relative risk [RR] = 1.33, 95% confidence interval [CI]: 1.171.52), which is to be expected based on the NHLBI guidelines that promote the superiority of ICS/SAL. However, the authors also found that decreased odds of ED visits and/or hospitalizations were observed with ICS/MON (adjusted odds ratio [OR] = 0.58; 95% CI: 0.35-0.98) than ICS/SAL. Why would ICS/MON prevent more ED visits and hospitalizations, if ICS/MON required more SABA use and was not recommended based on the preponderance of guideline evidence? The abstract did not provide more details, but, if the results do not fit expectations, either the reference NHBLI guidelines may be flawed or the propensity model created for the study may need correction. Upon further reading, 2 key points in the tables are found that may partially explain the contradiction. First, Table 5 illustrates that there was no difference in the ED/hospital rate for the 815 patients who had been treated with ICS previously. Only in the smaller group of 401 patients without prior ICS use was a significant decrease in ED/hospital rates found for the ICS/MON group. The difference was so profound (adjusted OR = 0.25; 95% CI, 0.08-0.79) that it caused the total study population to show a difference on this key patient-

inhaled corticosteroids or medium-dose inhaled corticosteroids as monotherapy." The outcomes tracked by the authors were oral corticosteroid (OCS) fill rates, short-acting beta-agonist (SABA) fill rates, visits to the emergency department (ED), and asthma-related hospitalizations. Since these outcomes are patient-oriented, and SABA overuse is a common marker used by clinicians to monitor for poor disease control, the outcomes are relevant and well chosen. When reviewing the outcome results, a significant discrepancy is immediately noted. The SABA fill rate was significantly higher among ICS/MON patients than ICS/SAL (adjusted relative risk [RR] = 1.33, 95% confidence interval [CI]: 1.17-1.52), which is to be expected based on the NHLBI guidelines that promote the superiority of ICS/SAL. However, the authors also found that decreased odds of ED visits and/or hospitalizations were observed with ICS/MON (adjusted odds ratio [OR] = 0.58; 95% CI: 0.35-0.98) than ICS/SAL. Why would ICS/MON prevent more ED visits and hospitalizations, if ICS/MON required more SABA use and was not recommended based on the preponderance of guideline evidence? The abstract did not provide more details, but, if the results do not fit expectations, either the reference NHBLI guidelines may be flawed or the propensity model created for the study may need correction. Upon further reading, 2 key points in the tables are found that may partially explain the contradiction.
First, Table 5 illustrates that there was no difference in the ED/hospital rate for the 815 patients who had been treated with ICS previously. Only in the smaller group of 401 patients without prior ICS use was a significant decrease in ED/hospital rates found for the ICS/MON group. The difference was so profound (adjusted OR = 0.25; 95% CI, 0.08-0.79) that it caused the total study population to show a difference on this key patient-  oriented outcome. The distinction that ICS/MON had lower ED/hospital rates in patients without prior ICS use is potentially explained by treatment bias; patients who received MON were likely considered to have less-severe disease based on NHLBI disease classification tables and were therefore less likely to later require emergent care. The second key point to potentially explain lower ED/ hospital rates in the ICS/MON group is the failure to properly categorize disease severity when building the propensity model. Originally buried in the footnotes of Table 1 was the notation that the disease severity covariate was factored into the model as a binary variable: patients were either considered mild intermittent or persistent (mild, moderate, and severe). Overlumping all forms of persistent disease into a single category when they have different treatment regimens and risk for ED/hospital makes the ICS/MON results questionable.

PP-ICONS
During the editing process, the authors attempted a post hoc classification of disease severity, based on a nonvalidated model from an abstract presented at a conference but otherwise unpublished. 5 (The proxy 4-group classification scheme was not used to construct the propensity model.) Since the model has never been tested against the NHLBI guidelines, the 4 "severity groups" are only estimates of true disease severity. If the estimates are assumed to reflect NHLBI categorization, then only 25% of the study patients should have been included; only patients in class 3 (moderate persistent asthma) or higher generally need add-on therapy with MON or SAL. 5 So, do the 75% of patients included with mild asthma not requiring add-on medications skew the study results showing that ICS/MON lowers the odds of ED/hospital visits? The model design does not measure severity in a way that allows such questions to be answered.
Finally, the number of patients and statistics chosen must be assessed. Including 1,216 patients would typically be considered a large study, but the authors did not discuss why they chose the number 1,216 or if a prestudy power calculation was performed. Using ORs with 95% CIs is standard practice and appropriate for this type of retrospective study.
In conclusion, PP-ICONS has been introduced as a tool for skimming abstracts and selecting relevant studies for further reading. When applied to the study by Allen-Ramey et al., issues of concern were found for the comparator design (ICS/SAL) as well as the outcome put forth by the authors that ICS/MON lowered ED and hospitalization rates. Unidentified treatment bias and failure to adequately classify disease severity may have skewed the results favoring montelukast.

DISCLOSURE
The author is a board-certified family physician assigned to Eglin AFB Florida, where he serves as Family Medicine Residency program director, HQ Air Armament Center. The opinions and assertions contained herein are the private views of the author and are not to be construed as official or as reflecting the views of any organization, including the U.S. Air Force medical department or the U.S. Air Force. He discloses no potential bias or conflict of interest relating to this editorial.